Method and apparatus for video error concealment in multi-view coded video using high level syntax

ABSTRACT

There are provided a method and apparatus for video error concealment in multi-view coded video using high level syntax. The apparatus includes a decoder for decoding pictures for at least one view corresponding to multi-view video content from a bitstream. The pictures are representative of at least a portion of a video sequence. At least some of the pictures correspond to different time instances in the video sequence. The decoder determines whether any of the pictures corresponding to a particular one of the different time instances are lost using an existing syntax element. The existing syntax element is for indicating a number of coded views in the bitstream, including the at least one view.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit, under 35 U.S.C. §365 ofInternational Application PCT/US2008/000115, filed Jan. 4, 2008 whichwas published in accordance with PCT Article 21(2) on Jul. 17, 2008 inEnglish and which claims the benefit of U.S. provisional patentapplication No. 60/883,454 filed Jan. 4, 2007.

TECHNICAL FIELD

The present principles relate generally to video encoding and decodingand, more particularly, to a method and apparatus for video errorconcealment in multi-view coded video using high level syntax.

BACKGROUND

When a picture is lost in a corrupted bitstream, several picture-basederror concealment methods can be used to conceal the lost picture. Inorder to perform concealment, the loss of a picture and the location ofthe picture have to be determined.

There have been several methods to detect the loss of a picture in thesingle view case. In the International Organization forStandardization/International Electrotechnical Commission (ISO/IEC)Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding(AVC) standard/International Telecommunication Union, TelecommunicationSector (ITU-T) H.264 recommendation (hereinafter the “MPEG-4 AVCstandard”), the concept of frame_num serves the purpose of detecting theloss of reference pictures. Additionally, Supplemental EnhancementInformation (SEI) messages such as the recovery point SEI message,sub-sequence SEI message, recovery point SEI message, reference picturemarking repetition SEI message, as well as the picture order count (POC)design, and the multiple reference picture buffering may be used for thepurpose of picture loss detection.

However, such methods have not been extended for the multi-view case.

SUMMARY

These and other drawbacks and disadvantages of the prior art areaddressed by the present principles, which are directed to a method andapparatus for video error concealment in multi-view coded video usinghigh level syntax.

According to an aspect of the present principles, there is provided anapparatus. The apparatus includes a decoder for decoding pictures for atleast one view corresponding to multi-view video content from abitstream. The pictures are representative of at least a portion of avideo sequence. At least some of the pictures correspond to differenttime instances in the video sequence. The decoder determines whether anyof the pictures corresponding to a particular one of the different timeinstances are lost using an existing syntax element. The existing syntaxelement is for indicating a number of coded views in the bitstream,including the at least one view.

According to another aspect of the present principles, there is provideda method. The method includes decoding pictures for at least one viewcorresponding to multi-view video content from a bitstream. The picturesare representative of at least a portion of a video sequence. At leastsome of the pictures correspond to different time instances in the videosequence. The decoding step includes determining whether any of thepictures corresponding to a particular one of the different timeinstances are lost using an existing syntax element. The existing syntaxelement is for indicating a number of coded views in the bitstream,including the at least one view.

These and other aspects, features and advantages of the presentprinciples will become apparent from the following detailed descriptionof exemplary embodiments, which is to be read in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The present principles may be better understood in accordance with thefollowing exemplary figures, in which:

FIG. 1 is a block diagram for an exemplary Multi-view Video Coding (MVC)decoder to which the present principles may be applied, in accordancewith an embodiment of the present principles;

FIG. 2 is a diagram for a time-first coding structure for a multi-viewvideo coding system with 8 views to which the present principles may beapplied, in accordance with an embodiment of the present principles;

FIG. 3 is a flow diagram for an exemplary method for decoding video datacorresponding to a video sequence using error concealment for lostpictures, in accordance with an embodiment of the present principles;and

FIG. 4 is a flow diagram for another exemplary method for decoding videodata corresponding to a video sequence using error concealment for lostpictures, in accordance with an embodiment of the present principles.

DETAILED DESCRIPTION

The present principles are directed to a method and apparatus for videoerror concealment in multi-view coded video using high level syntax.

The present description illustrates the present principles. It will thusbe appreciated that those skilled in the art will be able to devisevarious arrangements that, although not explicitly described or shownherein, embody the present principles and are included within its spiritand scope.

All examples and conditional language recited herein are intended forpedagogical purposes to aid the reader in understanding the presentprinciples and the concepts contributed by the inventor(s) to furtheringthe art, and are to be construed as being without limitation to suchspecifically recited examples and conditions.

Moreover, all statements herein reciting principles, aspects, andembodiments of the present principles, as well as specific examplesthereof, are intended to encompass both structural and functionalequivalents thereof. Additionally, it is intended that such equivalentsinclude both currently known equivalents as well as equivalentsdeveloped in the future, i.e., any elements developed that perform thesame function, regardless of structure.

Thus, for example, it will be appreciated by those skilled in the artthat the block diagrams presented herein represent conceptual views ofillustrative circuitry embodying the present principles. Similarly, itwill be appreciated that any flow charts, flow diagrams, statetransition diagrams, pseudocode, and the like represent variousprocesses which may be substantially represented in computer readablemedia and so executed by a computer or processor, whether or not suchcomputer or processor is explicitly shown.

The functions of the various elements shown in the figures may beprovided through the use of dedicated hardware as well as hardwarecapable of executing software in association with appropriate software.When provided by a processor, the functions may be provided by a singlededicated processor, by a single shared processor, or by a plurality ofindividual processors, some of which may be shared. Moreover, explicituse of the term “processor” or “controller” should not be construed torefer exclusively to hardware capable of executing software, and mayimplicitly include, without limitation, digital signal processor (“DSP”)hardware, read-only memory (“ROM”) for storing software, random accessmemory (“RAM”), and non-volatile storage.

Other hardware, conventional and/or custom, may also be included.Similarly, any switches shown in the figures are conceptual only. Theirfunction may be carried out through the operation of program logic,through dedicated logic, through the interaction of program control anddedicated logic, or even manually, the particular technique beingselectable by the implementer as more specifically understood from thecontext.

In the claims hereof, any element expressed as a means for performing aspecified function is intended to encompass any way of performing thatfunction including, for example, a) a combination of circuit elementsthat performs that function or b) software in any form, including,therefore, firmware, microcode or the like, combined with appropriatecircuitry for executing that software to perform the function. Thepresent principles as defined by such claims reside in the fact that thefunctionalities provided by the various recited means are combined andbrought together in the manner which the claims call for. It is thusregarded that any means that can provide those functionalities areequivalent to those shown herein.

Reference in the specification to “one embodiment” or “an embodiment” ofthe present principles means that a particular feature, structure,characteristic, and so forth described in connection with the embodimentis included in at least one embodiment of the present principles. Thus,the appearances of the phrase “in one embodiment” or “in an embodiment”appearing in various places throughout the specification are notnecessarily all referring to the same embodiment.

As used herein, “high level syntax” refers to syntax present in thebitstream that resides hierarchically above the macroblock layer. Forexample, high level syntax, as used herein, may refer to, but is notlimited to, syntax at the slice header level, the sequence parameter set(SPS) level, the picture parameter set (PPS) level, the view parameterset (VPS) level, the network abstraction layer (NAL) unit header level,and in a supplemental enhancement information (SEI) message.

For the sake of illustration and brevity, the following embodiments aredescribed herein with respect to the use of the sequence parameter set.However, it is to be appreciated that the present principles are notlimited to solely the use of the sequence parameter set with respect tothe improved signaling disclosed herein and, thus, such improvedsignaling may be implemented with respect to at least theabove-described types of high level syntaxes including, but not limitedto, syntaxes at the slice header level, the sequence parameter set (SPS)level, the picture parameter set (PPS) level, the view parameter set(VPS) level, the network abstraction layer (NAL) unit header level, andin a supplemental enhancement information (SEI) message, whilemaintaining the spirit of the present principles.

It is to be further appreciated that while one or more embodiments ofthe present principles are described herein with respect to the MPEG-4AVC standard, the present principles are not limited to solely thisstandard and, thus, may be utilized with respect to other video codingstandards, recommendations, and extensions thereof, including extensionsof the MPEG-4 AVC standard, while maintaining the spirit of the presentprinciples.

Moreover, it is to be appreciated that the use of the term “and/or”, forexample, in the case of “A and/or B”, is intended to encompass theselection of the first listed option (A), the selection of the secondlisted option (B), or the selection of both options (A and B). As afurther example, in the case of “A, B, and/or C”, such phrasing isintended to encompass the selection of the first listed option (A), theselection of the second listed option (B), the selection of the thirdlisted option (C), the selection of the first and the second listedoptions (A and B), the selection of the first and third listed options(A and C), the selection of the second and third listed options (B andC), or the selection of all three options (A and B and C). This may beextended, as readily apparent by one of ordinary skill in this andrelated arts, for as many items listed.

Turning to FIG. 1, an exemplary Multi-view Video Coding (MVC) decoder isindicated generally by the reference numeral 100. The decoder 100includes an entropy decoder 105 having an output connected in signalcommunication with an input of an inverse quantizer 110. An output ofthe inverse quantizer is connected in signal communication with an inputof an inverse transformer 115. An output of the inverse transformer 115is connected in signal communication with a first non-inverting input ofa combiner 120. An output of the combiner 120 is connected in signalcommunication with an input of a deblocking filter 125 and an input ofan intra predictor 130. An output of the deblocking filter 125 isconnected in signal communication with an input of a reference picturestore 140 (for view i). An output of the reference picture store 140 isconnected in signal communication with a first input of a motioncompensator 135.

An output of a reference picture store 145 (for other views) isconnected in signal communication with a first input of adisparity/illumination compensator 150.

An input of the entropy coder 105 is available as an input to thedecoder 100, for receiving a residue bitstream. Moreover, an input of amode module 160 is also available as an input to the decoder 100, forreceiving control syntax to control which input is selected by theswitch 155. Further, a second input of the motion compensator 135 isavailable as an input of the decoder 100, for receiving motion vectors.Also, a second input of the disparity/illumination compensator 150 isavailable as an input to the decoder 100, for receiving disparityvectors and illumination compensation syntax.

An output of a switch 155 is connected in signal communication with asecond non-inverting input of the combiner 120. A first input of theswitch 155 is connected in signal communication with an output of thedisparity/illumination compensator 150. A second input of the switch 155is connected in signal communication with an output of the motioncompensator 135. A third input of the switch 155 is connected in signalcommunication with an output of the intra predictor 130. An output ofthe mode module 160 is connected in signal communication with the switch155 for controlling which input is selected by the switch 155. An outputof the deblocking filter 125 is available as an output of the decoder.

In accordance with the present principles, methods and apparatus areprovided for video error concealment in multi-view coded video usinghigh level syntax. The present principles, at the least, address theproblem of detection of picture loss in the case of multi-view codedvideo. Methods and apparatus are provided herein to identify/detectwhich pictures of a view are missing, lost, or dropped duringtransmission of a multi-view coded video sequence.

In an error-prone transmission environment, such as the Internet,wireless networks, and so forth, a transmitted video bitstream maysuffer corruptions caused by, for example, channel impairment. A commonsituation encountered in some practical systems is that certaincompressed video pictures are dropped from a bitstream. This isespecially true for low bit-rate applications where a picture is smallenough to be coded into a transmit unit, such as a real-time transportprotocol (RTP) packet. At the receiver end, a robust video decodershould be able to detect such losses in order to conceal them.

In multi-view video coding (MVC), there are several views present in thecoded video sequence. In the case of the current MVC extension of theMPEG-4 AVC Standard, each picture has associated with it a viewidentifier to identify which view to which it belongs. TABLE 1 shows theNetwork Abstraction Layer (NAL) unit header for the scalable videocoding (SVC) multi-view video coding (MVC) extension syntax.Additionally, there are several high level syntaxes (in addition to theMPEG-4 AVC Standard syntaxes) that are present to assist in the decodingof the pictures from different views. These syntaxes are present in theSequence Parameter Set (SPS) extension. TABLE 2 shows the sequenceparameter set (SPS) in the multi-view video coding (MVC) extension ofthe MPEG-4 AVC Standard.

TABLE 1 nal_unit_header_svc_mvc_extension( ) { C Descriptor svc_mvc_flagAll u(1) if (!svc_mvc_flag) { priority_id All u(6) discardable_flag Allu(1) temporal_level All u(3) dependency_id All u(3) quality_level Allu(2) layer_base_flag All u(1) use_base_prediction_flag All u(1)fragmented_flag All u(1) last_fragment_flag All u(1) fragment_order Allu(2) reserved_zero_two_bits All u(2) } else { temporal_level All u(3)view_level All u(3) anchor_pic_flag All u(1) view_id All u(10) idr_flagAll u(1) reserved_zero_five_bits All u(5) } nalUnitHeaderBytes += 3 }

TABLE 2 seq_parameter_set_mvc_extension( ) { C Descriptornum_views_minus_1 ue(v) for(i = 0; i <= num_views_minus_1; i++) {num_anchor_refs_I0[i] ue(v) for( j = 0; j < num_anchor_refs_I0[i]; j++ )anchor_ref_I0[i][j] ue(v) num_anchor_refs_I1[i] ue(v) for( j = 0; j <num_anchor_refs_I1[i]; j++ ) anchor_ref_I1[i][j] ue(v) } for(i = 0; i <=num_views_minus_1; i++) { num_non_anchor_refs_I0[i] ue(v) for( j = 0; j< num_non_anchor_refs_I0[i]; j++ ) non_anchor_ref_I0[i][j] ue(v)num_non_anchor_refs_I1[i] ue(v) for( j = 0; j <num_non_anchor_refs_I1[i]; j++ ) non_anchor_ref_I1[i][j] ue(v) } }

Thus, the current proposal for multi-view video coding based on theMPEG-4 AVC Standard (hereinafter “current MVC proposal for MPEG-4 AVC)includes high level syntax in the sequence parameter set (SPS) toindicate the number of coded views in the sequence. Additionally, thecurrent MVC proposal for MPEG-4 AVC includes the inter-view referencesinformation for a view. The current MVC proposal for MPEG-4 AVC furtherdistinguishes the dependencies of the anchor and non-anchor picture byseparately sending the reference view identifiers. This is shown inTABLE 2, which includes information of which views are used as areference for a certain view. We have recognized and propose that thisinformation (the number of coded views) can be used in order to detectpicture loss in the case of multi-view coded video.

In the current multi-view video coding (MVC) extension of the MPEG-4 AVCStandard, it is mandated that pictures that belong to a certain timeinstant are coded first for all the views. Turning to FIG. 2, atime-first coding structure for a multi-view video coding system with 8views is indicated generally by the reference numeral 200. In theexample of FIG. 2, all pictures at the same time instance from differentviews are coded contiguously. Thus, all pictures (S0-S7) at time instantT0 are coded first, followed by pictures (S0-S7) at time T8, and so on.This is called time-first coding. Timely concealment of the lost pictureis critical for the objective quality of other views.

Knowing this (time-first coding) and also the number of coded views inthe sequence from the sequence parameter set (SPS) we can detect theloss of a picture. Additionally, in the illustrated embodiment, wepresume that all the views have the same frame rate. However, it is tobe appreciated that the present principles are not solely limited toapplications involving video having the same frame rate and, thus, mayalso be readily applied to application involving video with differentframe rates, while maintaining the spirit of the present principles. Theframe rate information is present in the Video Usability Information(VUI) parameters present in the sequence parameter set. Since theseparameters are optional they may or may not be present. In the case theyare present, a separate sequence parameter set needs to be signaled forthe view with a different frame rate. Once the decoder knows of the viewthat has a different frame rate, it may invoke an error detection methodor simply disregard the outcome of the error detection method.Additionally, the decoder may choose to conceal the picture, but not usethe picture for display purposes.

An embodiment of this detection algorithm is as follows. From thesequence parameter set we know the number of views that are coded(num_views_minus_(—)1). We also know that all pictures at a certain timeare coded first (time-first coding). Thus, for each time instance, wecan first buffer all the pictures from each view (e.g., time T0). Wekeep a count of the number of pictures that arrive for that timeinstance (N) and the view identifier (view_id) of the pictures. Since weknow the number of coded views from the sequence parameter set syntax,we compare these values to determine if there was any loss. If N is lessthan the value indicated in the SPS then we know that there was a loss.Additionally, since we keep track of the view_id that have arrived wecan determine which view_ids are missing.

Also, if we assume that the MPEG-4 AVC Standard compatible view is thefirst view that is coded, then we know that in between two MPEG-4 AVCStandard compatible NAL units we are expecting num_views_minus_(—)1views. This information can also be used to detect the loss of pictures.Similar to the above embodiment, we can then determine the missingview_id(s).

In another embodiment, we only need to perform the buffering once at thebeginning of the bitstream or after a change of coding order. Once wehave determined the coding order by looking at the view_id order duringthe first buffering period, for the subsequent time instance we onlyneed to check if the picture that arrives has the expected view_id. Ifit is determined that it does not have the expect view_id then we knowthat a picture belonging to the expected view_id was lost and thus needsto be concealed before decoding of other pictures.

Turning to FIG. 3, an exemplary method for decoding video datacorresponding to a video sequence using error concealment for lostpictures is indicated generally by the reference numeral 300.

The method 300 includes a start block 305 that passes control to afunction block 310. The function block 310 parses the sequence parameterset (SPS), the picture parameter set (PPS), the view parameter set(VPS), the network abstraction layer (NAL) unit header, and/or anysupplemental enhancement information (SEI) messages, and passes controlto a function block 315. The function block 315 sets a variable NumViewsequal to num_view_minus1+1, sets a variable PrevPOC equal to zero, setsa variable RecvPic equal to zero, and passes control to a decision block320. The decision block 320 determines whether or not the end of thevideo sequence has been reached. If so, then control is passed to an endblock 399. Otherwise, control is passed to a function block 325.

The function block 325 reads the picture order count (POC) of the nextpictures, increments the variable RecvPic, and passes control to adecision block 330. The decision block 330 determines whether or not thevariable CurrPOC is equal to the variable PrevPOC. If so, the control ispassed to a function block 335. Otherwise, control is passed to afunction block 340.

The function block 335 buffers the current picture, and returns controlto the function block 325.

The function block 340 buffers the current picture, sets the variablePrevPOC equal to the variable CurrPOC, and passes control to thedecision block 345. The decision block 345 determines whether or not thevariable RecvPic is equal to the variable NumViews. If so, then controlis passed to a decision block 355. Otherwise, control is passed to afunction block 350.

The decision block 355 determines whether or not the variable RecvPic isequal to zero. If so, then control is passed to a function block 360.Otherwise, control is passed to a function block 370.

The function block 360 parses the next picture, and passes control to afunction block 365. The function block 365 decodes the current picture,and returns control to the decision block 355.

The function block 350 checks and stores the missing view_ids, andpasses control to the decision block 355.

The function block 370 conceals the pictures corresponding to themissing view_ids, and returns control to the decision block 320.

Turning to FIG. 4, another exemplary method for decoding video datacorresponding to a video sequence using error concealment for lostpictures is indicated generally by the reference numeral 400.

The method 400 includes a start block 405 that passes control to afunction block 410. The function block 410 parses the sequence parameterset (SPS), the picture parameter set (PPS), the view parameter set(VPS), the network abstraction layer (NAL) unit header, and/or anysupplemental enhancement information (SEI) messages, and passes controlto a function block 415. The function block 415 sets a variable NumViewsequal to num_view_minus1+1, sets a variable PrevPOC equal to zero, setsa variable RecvPic equal to zero, sets a variable ViewCodingOrder equalto zero, and passes control to a decision block 420. The decision block420 determines whether or not the end of the video sequence has beenreached. If so, then control is passed to an end block 499. Otherwise,control is passed to a function block 425.

The function block 425 reads the picture order count (POC) of the nextpictures, increments the variable RecvPic, and passes control to adecision block 430. The decision block 430 determines whether or not thevariable CurrPOC is equal to the variable PrevPOC. If so, the control ispassed to a decision block 435. Otherwise, control is passed to afunction block 450.

The decision block 435 determines whether or not the variableViewCodingOrder is equal to one. If so, then control is passed to adecision block 440. Otherwise, control is passed to a function block485.

The decision block 440 determines whether or not concealment isrequired. If so, then control is passed to a function block 445.Otherwise, control is passed to a function block 490.

The function block 445 conceals the pictures of the missing view_ids,and returns control to a decision block 420.

The function block 450 buffers the current picture, sets the variablePrevPoc equal to the variable CurrPOC, and passes control to thedecision block 455. The decision block 455 determines whether thevariable RecvPic is equal to the variable NumViews. If so, then controlis passed to a function block 460. Otherwise, control is passed to afunction block 480.

The function block 460 stores the view coding order, setsViewCodingOrder equal to one, and passes control to a decision block465. The decision block 465 determines whether or not RecvPic is equalto zero. If so, then control is passed to a function block 470.Otherwise, control is passed to a function block 445.

The function block 470 parses the next picture, and passes control to afunction block 475. The function block 475 decodes the current picture,and returns control to the decision block 465.

The function block 485 buffers the current picture, and returns controlto a function block 425.

The function block 480 checks and stores missing view_ids, and passescontrol to the decision block 465.

The function block 490 decodes the current picture, and returns controlto the decision block 420.

A description will now be given of some of the many attendantadvantages/features of the present invention, some of which have beenmentioned above. For example, one advantage/feature is an apparatus thatincludes a decoder for decoding pictures for at least one viewcorresponding to multi-view video content from a bitstream. The picturesare representative of at least a portion of a video sequence. At leastsome of the pictures correspond to different time instances in the videosequence. The decoder determines whether any of the picturescorresponding to a particular one of the different time instances arelost using an existing syntax element. The existing syntax element isfor indicating a number of coded views in the bitstream, including theat least one view.

Another advantage/feature is the apparatus having the decoder asdescribed above, wherein the existing syntax element is a multi-viewvideo coding syntax element.

Yet another advantage/feature is the apparatus having the decoderwherein the existing syntax element is a multi-view video coding syntaxelement as described above, wherein the multi-view video coding syntaxelement corresponds to an extension of the International Organizationfor Standardization/International Electrotechnical Commission MovingPicture Experts Group-4 Part 10 Advanced Video Codingstandard/International Telecommunication Union, Telecommunication SectorH.264 recommendation.

Still another advantage/feature is the apparatus having the decoder asdescribed above, wherein the existing syntax element is present at ahigh level.

Moreover, another advantage/feature is the apparatus having the decoderas described above, wherein the high level corresponds to at least atone of a slice header level, a sequence parameter set level, a pictureparameter set level, a view parameter set level, a network abstractionlayer unit header level, and a level corresponding to a supplementalenhancement information message.

Further, another advantage/feature is the apparatus having the decoderas described above, wherein any of the pictures corresponding to theparticular one of the different time instances are buffered, and thedecoder maintains a count for the pictures that arrive at the particularone of the different time instances.

Also, another advantage/feature is the apparatus having the decoderwherein any of the pictures corresponding to the particular one of thedifferent time instances are buffered, and the decoder maintains a countfor the pictures that arrive at the particular one of the different timeinstances as described above, wherein the decoder compares a value ofthe count to a value of the existing syntax element.

Additionally, another advantage/feature is the apparatus having thedecoder that compares a value of the count to a value of the existingsyntax element as described above, wherein a particular one of thepictures is designated as lost, when the value of the count for theparticular picture does not equal the value of the existing syntaxelement.

Moreover, another advantage/feature is the apparatus having the decoderwherein a particular one of the pictures is designated as lost, when thevalue of the count for the particular picture does not equal the valueof the existing syntax element as described above, wherein the decoderdetermines a view identifier of the particular one of the picturesdesignated as lost by looking for a missing view identifier from among aset of view identifiers that have arrived corresponding to thebitstream.

Further, another advantage/feature is the apparatus having the decoderas described above, wherein to decode the pictures from the bitstream,the decoder buffers the pictures only once.

Also, another advantage/feature is the apparatus having the decoder thatbuffers the pictures only once to decode the pictures from the bitstreamas described above, wherein the decoder buffers the pictures only once,at the beginning of the bitstream.

Additionally, another advantage/feature is the apparatus having thedecoder that buffers the pictures only once to decode the pictures fromthe bitstream as described above, wherein the decoder buffers thepictures only once, subsequent to a change of coding order.

Moreover, another advantage/feature is the apparatus having the decoderthat buffers the pictures only once to decode the pictures from thebitstream as described above, wherein the decoder maintains an order ofview identifiers for the pictures, subsequent to the buffering.

Further, another advantage/feature is the apparatus having the decoderthat maintains an order of view identifiers for the pictures, subsequentto the buffering, as described above, wherein the decoder uses the orderthat is maintained for subsequent ones of the different time instancesto determine if any of the pictures corresponding to the subsequent onesof the different time instances are lost, the subsequent ones of thedifferent time instances being subsequent with respect to the particularone of the different time instances.

Also, another advantage/feature is the apparatus having the decoder asdescribed above, wherein the at least one view comprises at least twoviews, and the decoder uses a number of views between the two views todetermine if any of the pictures are lost, when a first coded view fromamong the at least one view, and the two views, are compatible viewswith respect to the International Organization forStandardization/International Electrotechnical Commission Moving PictureExperts Group-4 Part 10 Advanced Video Coding standard/internationalTelecommunication Union, Telecommunication Sector H.264 recommendation,the first coded view being one of one of the two views or another view.

These and other features and advantages of the present principles may bereadily ascertained by one of ordinary skill in the pertinent art basedon the teachings herein. It is to be understood that the teachings ofthe present principles may be implemented in various forms of hardware,software, firmware, special purpose processors, or combinations thereof.

Most preferably, the teachings of the present principles are implementedas a combination of hardware and software. Moreover, the software may beimplemented as an application program tangibly embodied on a programstorage unit. The application program may be uploaded to, and executedby, a machine comprising any suitable architecture. Preferably, themachine is implemented on a computer platform having hardware such asone or more central processing units (“CPU”), a random access memory(“RAM”), and input/output (“I/O”) interfaces. The computer platform mayalso include an operating system and microinstruction code. The variousprocesses and functions described herein may be either part of themicroinstruction code or part of the application program, or anycombination thereof, which may be executed by a CPU. In addition,various other peripheral units may be connected to the computer platformsuch as an additional data storage unit and a printing unit.

It is to be further understood that, because some of the constituentsystem components and methods depicted in the accompanying drawings arepreferably implemented in software, the actual connections between thesystem components or the process function blocks may differ dependingupon the manner in which the present principles are programmed. Giventhe teachings herein, one of ordinary skill in the pertinent art will beable to contemplate these and similar implementations or configurationsof the present principles.

Although the illustrative embodiments have been described herein withreference to the accompanying drawings, it is to be understood that thepresent principles is not limited to those precise embodiments, and thatvarious changes and modifications may be effected therein by one ofordinary skill in the pertinent art without departing from the scope orspirit of the present principles. All such changes and modifications areintended to be included within the scope of the present principles asset forth in the appended claims.

The invention claimed is:
 1. An apparatus, comprising: a decoder fordecoding pictures for at least one view corresponding to multi-viewvideo content from a bitstream, the pictures representative of at leasta portion of a video sequence, at least some of the picturescorresponding to different time instances in the video sequence, whereinsaid decoder determines whether any of the pictures corresponding to aparticular one of the different time instances are lost using anexisting syntax element, the existing syntax element for indicating anumber of coded views in the bitstream, including the at least one view.2. The apparatus of claim 1, wherein the existing syntax element is amulti-view video coding syntax element.
 3. The apparatus of claim 2,wherein the multi-view video coding syntax element corresponds to anextension of the International Organization forStandardization/International Electrotechnical Commission Moving PictureExperts Group-4 Part 10 Advanced Video Coding standard/InternationalTelecommunication Union, Telecommunication Sector H.264 recommendation.4. The apparatus of claim 1, wherein the existing syntax element ispresent at a high level.
 5. The apparatus of claim 1, wherein the highlevel corresponds to at least at one of a slice header level, a sequenceparameter set level, a picture parameter set level, a view parameter setlevel, a network abstraction layer unit header level, and a levelcorresponding to a supplemental enhancement information message.
 6. Theapparatus of claim 1, wherein any of the pictures corresponding to theparticular one of the different time instances are buffered, and saiddecoder maintains a count for the pictures that arrive at the particularone of the different time instances.
 7. The apparatus of claim 6,wherein said decoder compares a value of the count to a value of theexisting syntax element.
 8. The apparatus of claim 7, wherein aparticular one of the pictures is designated as lost, when the value ofthe count for the particular picture does not equal the value of theexisting syntax element.
 9. The apparatus of claim 8, wherein saiddecoder determines a view identifier of the particular one of thepictures designated as lost by looking for a missing view identifierfrom among a set of view identifiers that have arrived corresponding tothe bitstream.
 10. The apparatus of claim 1, wherein to decode thepictures from the bitstream, said decoder buffers the pictures onlyonce.
 11. The apparatus of claim 10, wherein said decoder buffers thepictures only once, at the beginning of the bitstream.
 12. The apparatusof claim 10, wherein said decoder buffers the pictures only once,subsequent to a change of coding order.
 13. The apparatus of claim 10,wherein said decoder maintains an order of view identifiers for thepictures, subsequent to the buffering.
 14. The apparatus of claim 13,wherein said decoder uses the order that is maintained for subsequentones of the different time instances to determine if any of the picturescorresponding to the subsequent ones of the different time instances arelost, the subsequent ones of the different time instances beingsubsequent with respect to the particular one of the different timeinstances.
 15. The apparatus of claim 1, wherein the at least one viewcomprises at least two views, and said decoder uses a number of viewsbetween the two views to determine if any of the pictures are lost, whena first coded view from among the at least one view, and the two views,are compatible views with respect to the International Organization forStandardization/International Electrotechnical Commission Moving PictureExperts Group-4 Part 10 Advanced Video Coding standard/InternationalTelecommunication Union, Telecommunication Sector H.264 recommendation,the first coded view being one of one of the two views or another view.16. A method, comprising: decoding pictures for at least one viewcorresponding to multi-view video content from a bitstream, the picturesrepresentative of at least a portion of a video sequence, at least someof the pictures corresponding to different time instances in the videosequence, wherein said decoding step comprises determining whether anyof the pictures corresponding to a particular one of the different timeinstances are lost using an existing syntax element, the existing syntaxelement for indicating a number of coded views in the bitstream,including the at least one view.
 17. The method of claim 16, wherein theexisting syntax element is a multi-view video coding syntax element. 18.The method of claim 17, wherein the multi-view video coding syntaxelement corresponds to an extension of the International Organizationfor Standardization/International Electrotechnical Commission MovingPicture Experts Group-4 Part 10 Advanced Video Codingstandard/International Telecommunication Union, Telecommunication SectorH.264 recommendation.
 19. The method of claim 16, wherein the existingsyntax element is present at a high level.
 20. The method of claim 16,wherein the high level corresponds to at least at one of a slice headerlevel, a sequence parameter set level, a picture parameter set level, aview parameter set level, a network abstraction layer unit header level,and a level corresponding to a supplemental enhancement informationmessage.
 21. The method of claim 16, wherein any of the picturescorresponding to the particular one of the different time instances arebuffered, and said decoding step comprises maintaining a count for thepictures that arrive at the particular one of the different timeinstances.
 22. The method of claim 21, wherein said decoding stepcomprises comparing a value of the count to a value of the existingsyntax element.
 23. The method of claim 22, wherein a particular one ofthe pictures is designated as lost, when the value of the count for theparticular picture does not equal the value of the existing syntaxelement.
 24. The method of claim 23, wherein said decoding stepcomprises determining a view identifier of the particular one of thepictures designated as lost by looking for a missing view identifierfrom among a set of view identifiers that have arrived corresponding tothe bitstream.
 25. The method of claim 16, wherein to decode thepictures from the bitstream, said decoding step comprises buffering thepictures only once.
 26. The method of claim 25, wherein said decoderbuffers the pictures only once, at the beginning of the bitstream. 27.The method of claim 25, wherein said decoder buffers the pictures onlyonce, subsequent to a change of coding order.
 28. The method of claim25, wherein said decoding step comprises maintaining an order of viewidentifiers for the pictures, subsequent to the buffering.
 29. Themethod of claim 28, wherein said decoding step comprising using theorder that is maintained for subsequent ones of the different timeinstances to determine if any of the pictures corresponding to thesubsequent ones of the different time instances are lost, the subsequentones of the different time instances being subsequent with respect tothe particular one of the different time instances.
 30. The method ofclaim 16, wherein the at least one view comprises at least two views,and said decoding step uses a number of views between the two views todetermine if any of the pictures are lost, when a first coded view fromamong the at least one view, and the two views, are compatible viewswith respect to the International Organization forStandardization/International Electrotechnical Commission Moving PictureExperts Group-4 Part 10 Advanced Video Coding standard/InternationalTelecommunication Union, Telecommunication Sector H.264 recommendation,the first coded view being one of one of the two views or another view.